The enterprise computing landscape has recently undergone a fundamental shift in storage architectures in which the central-service architecture has given way to distributed storage systems. Distributed storage systems built from commodity computer systems can deliver high performance, availability, and scalability for new data-intensive applications at a fraction of cost compared to monolithic disk arrays. To unlock the full potential of distributed storage systems, data is replicated across multiple instances of the distributed storage system at different geographical locations, thereby increasing availability and reducing network distance from clients.
In a distributed storage system, objects are dynamically created and deleted in different instances of the distributed storage system. However, different replication requests may have different priorities. It is important to execute replication requests in priority order so as to replicate the more important objects first. For example, a newly uploaded object has just one replica. Thus, it is more important to create replicas of the new object before creating replicas of existing objects that already has a plurality of replicas in order to minimize the probability of data loss in the new object. Another example is a video that becomes a hit over night. In this case, the number of replicas of the video needs to be increased as soon as possible in order to handle the increased demand. Therefore, it is desirable to properly prioritize replication requests and execute them in a timely fashion while sustaining very high loads.
In a small-scale distributed storage system, managing replicas of objects is a tractable problem. However, there are no existing techniques for managing replicas of objects in a planet-wide distributed storage system that includes trillions of objects, petabytes of data, dozens of data centers across the planet.